This invention relates to methods and systems for automatically generating reference height data for use in a three-dimensional inspection system. The invention is particularly well adapted for use in 3D inspection systems for solder paste and component inspection where 3D data is to be obtained from at least one reference region of the circuit. A baseline (zero) reference height data is used to calculate the height of paste, leads, or components. Other applicable calculations and measurements include area, volume, x and y position and orientation.
Over the past several years, the use of 3D inspection for solder paste, components, microelectronic assemblies etc. has proven to be beneficial for both process control and random defect detection. In particular, 3D systems have shown capability of correlating defects at the end of line (i.e., electrical test) with solder paste volume. Hence, 3D process control can be used to maximize yield. Unfortunately, because board warpage is on the order of the inspection tolerance, and additional stackups include tilt and positioning of the board, some method of finding a baseline local reference plane or surface is needed.
The data to provide such a reference may be obtained from bare or covered conductor traces, pads, 3D fiducials, board fiducials, ground planes, or fiberglass backgrounds. Specially fabricated opaque 3D fiducials which have built-in contrast and predetermined locations, would perhaps be ideal but are not practical in many cases due to retooling cost. Therefore, data from the circuit boards, the configurations of which vary greatly, is needed.
SMT boards are being produced with increasing density and, when vision systems are utilized for inspection, time-consuming setup procedures are often required to xe2x80x9cteachxe2x80x9d the inspection locations and criteria for 3D inspection. In contract manufacturing environments, where changeover is frequent, time-consuming setup reduces the value of the vision system in the overall process. In contract manufacturing environments changeover is so frequent that use of CAD data and supplementary data like Gerber files becomes unwieldy due in part to limited availability of such information.
Although the use of CAD download to specify interconnect locations (i.e., chip pad sites) has mitigated the problem to some degree, the CAD data often lack specific, or at least the best, information needed for 3D inspection. 3D inspection and measurement systems require a height reference for accurately determining the height, area, and volume of solder paste and/or lead or component height, presence/absence, and position (x,y, and orientation). For example, it is desirable to place inspection windows on bare conductor pad regions in close proximity to a paste deposit, but this is rarely possible because available pad areas will be covered by paste. In order to define an accurate height reference plane at the feature to be inspected, numerous reference sites surrounding the feature need to be defined. Special board designs to place fiducials at predetermined locations are possible, but rarely practical because of board re-tooling and design costs. As a result, traces on the board, which may be covered by solder mask, are often manually specified by the user to provide reference height information. Manually identifying reference points to do 100% inspection of a large (18xe2x80x3xc3x9718xe2x80x3) board can take several hours. Contract manufacturers require the programming of such a board to be less than 30 minutes, and preferably much less.
A system which provides the benefit of three-dimensional inspection, which is preferably 100% of every board, yet provides a method for fast, automatic learning, is advantageous. Built-in error detection is particularly advantageous in contract PCB manufacturing environments where frequent changeover is required and many board designs are produced in relatively low volume.
Mengel, P., xe2x80x9cAutomated Inspection of Solder Joints on PC Boards By Supplementary Processing of 3D and Gray-Level Imagesxe2x80x9d, IECON 1990, November 1990 discloses the use of 3D and gray-level information for PCB inspection, including the use of CAD/CAM data for specifying the geometric positions and tolerances of components. In the absence of CAD data, a xe2x80x9cteach-inxe2x80x9d on a sample PCB is performed. xe2x80x9cAn interactive interfacexe2x80x9d is disclosed which permits correction or optimization of testing instructions for special format devices.
U.S. Pat. No. 5,088,828, Doemens et al., discloses the use of evaluation windows on a PCB for recognizing defects of interconnects and defines zero datum. A triangulation-based 3D sensor is used for measurement. Also disclosed is the use of a detector for fluorescent light arising upon incidence of the laser beam on the insulating material. The invention includes a step of comparing the dimensions of the interconnect with stored spacing criteria by identifying the width or areas having a height of zero. Scan fields offset with respect to one another are used to determine warping.
U.S. Pat. No. 5,450,204, Shigeyama et al., discloses a 3D inspection machine for measuring the position, area, and thickness of amount of creamed solder and comparing with stored reference data. The sensor is effectively a phased shifted 3D Moire system. An inspection file is produced by downloading CAD data for a screen mask.
U.S. Pat. No. 5,822,449, Kobayashi et al., discloses a method for teaching without exclusively using the inspection machine. Data about pictures and decision criteria are stored and composed, and by being externally obtained, avoids manual entry operation. An adjustment step is provided for parameters including the positions of components, data of lands, bridge detection, and mounting qualities. Also, step ST17 is teaching modification to modify data which results if the xe2x80x9cimproper portion is found as the results of the automatic inspectionxe2x80x9d.
U.S. Pat. No. 5,465,152, Bilodeau et al., discloses the use of opaque fiducials as index pads (reference points) xe2x80x9cdesigned inxe2x80x9d at predetermined locations of a substrate for coplanarity and warpage determination. This approach generally must be included at the board design stage which would incur substantial tooling costs and delays.
U.S. Pat. No. 5,862,973, Wasserman, combines video camera inspection with structured light for solder paste detect detection and statistical process control.
U.S. Pat. No. 5,902,353, Raymond, discloses relative height measurement of components where xe2x80x9ckey pointsxe2x80x9d, which may be points on the board under test or on components, are used and the height measurement compared with a reference to determine faulty placement. The preferred height detection system disclosed is a triangulation-based PSD point detection system. Component-bycomponent programming, automatic or manual, can include parts list, component and board geometries, machine instructions, and solder paste stencil information. The use of a miniature video camera is suggested for manual fiducial alignment and for manually specifying key points for each component. The set of key points for component measurement preferably includes at least one reference.
None of the prior art references teach or disclose a method and system for automatic learning of reference points in a 3D inspection machine, wherein a machine vision subsystem automatically locates and analyzes image data for establishing a baseline height reference.
The primary object of the present invention is to provide a method and system for automatically generating reference height data for use in a three-dimensional inspection system.
In carrying out the above object and other objects of the present invention, a method is provided for automatically generating reference height data for use in a 3D inspection system. The method includes automatically determining positional location of at least one local reference area on an object separate from but relative to a predetermined site on the object.
Preferably, the step of automatically determining includes the steps of imaging the object to obtain image data and processing the image data to obtain the location of the at least one local reference area.
The object may be a printed circuit board and the predetermined site may be an interconnect site. The step of processing the image data may include the step of processing the image data with conductor coordinate information which represents location and/or connectivity of conductor runs separated from the interconnect site.
The step of sensing may include the step of imaging.
The predetermined site may be substantially coplanar to the at least one local reference area. Also, the step of sensing may include the step of sensing height of a plurality of portions of the at least one local reference area to obtain data and processing the data to obtain the corresponding signal.
Also, the step of automatically determining may automatically determine locations of a plurality of substantially planar local reference areas wherein the step of sensing senses height of at least a portion of the plurality of local reference areas and generates corresponding signals and wherein the step of processing processes the signals to obtain the reference height data.
Further in carrying out the above object and other objects of the present invention, a system is provided for automatically generating reference height data for use in a 3D inspection system. The system includes a machine vision subsystem for automatically determining location of at least one local reference area on an object separate from but relative to a predetermined site on the object. The system includes a 3D sensor for sensing height of at least a portion of the at least one local reference area on the object and generating a corresponding signal. Still further, the system includes a signal processor for processing the signal to obtain the reference height data.
The machine vision subsystem typically includes an imaging section to image the object to obtain image data and a data processing section for processing the image data to obtain the location of the at least one reference area. The imaging section may include a video camera.
The 3D sensor may be a triangulation sensor and preferably the triangulation sensor includes a laser scanner.
The object may be a printed circuit board and the predetermined site may be an interconnect site.
The predetermined site is preferably substantially coplanar to the at least one local reference area.
The 3D sensor may be part of the machine vision subsystem.
The 3D sensor may include a projector for projecting a point, line grid, or other pattern onto the local reference area.
Still further in carrying out the invention, a method and system are provided in which a height sensor acquires location information for establishing a height reference. The height reference is located relative to predetermined interconnect sites where paste is to be deposited or components placed, and from which the relative height of solder paste or components is to be determined using the reference heights during the subsequent inspection process. An algorithm determines the location of an appropriate set of reference point locations and designates at least the x,y coordinate information for later use by the inspection program.
The algorithm automatically determines from image data, acquired prior to executing an inspection algorithm, the position relative to the interconnect for placement of at least one reference box or areas from which the relative height of solder paste or components is to be determined.
A plurality of reference boxes may be for a reference surface, such as a plane, which substantially intersects the interconnect location in three dimensions.
The learning algorithm may be executed using a bare circuit board, a board with solder paste deposits, a board with components attached, and may be supplemented with CAD information.
The locations where the reference boxes are to be placed may generally be distant and disjoint from image regions where the interconnects are located.
In a preferred method and system, all the height reference locations on the circuit board are found automatically, but an alternative semi-automatic embodiment in which interaction with the operator through a user interface to guide the learning process is within the scope of the invention provided a subset of all height reference point locations are found without operator supervision.
Alternatively, a xe2x80x9clearn on the flyxe2x80x9d algorithm automatically determines from image data, acquired during the execution of the inspection program, the position relative to the interconnect for placement of reference boxes from which the relative height of solder paste or components is to be determined.
The xe2x80x9con the flyxe2x80x9d approach may avoid archival storage of the reference locations prior to an initial inspection. The reference data may be recalculated during the inspection process, or stored in memory.
The imaging system and/or computer used for the automatic learning phase may be physically separated from that of the inspection system, resulting in an xe2x80x9coff-linexe2x80x9d learning environment.
The algorithm may search a region in the proximity of an interconnect, for a conductor in close proximity to the interconnect, or a conductor connected to the interconnect.
Conductor surfaces are preferred for reference points, or traces covered with dielectric.
However, the algorithm may elect to use surfaces other than the conductors, for example, the fiberglass board surface if the corresponding height data is accurate.
The algorithm may store the height data from a reference box for later use and comparison with specifications.
The stored height data may be the average height obtained from height data in the reference box.
In contrast to the average height from the entire reference box, the calculated height may be from data points from a portion of the box, where the data points are obtained from a segmentation algorithm.
The algorithm may search the perimeter of the image for conductor traces; the conductor traces may run in the horizontal, vertical or diagonal directions.
The algorithm may include regions other than conductor traces, including, for example, ground planes, fiberglass, silk-screened areas, patterns which are laser marked or etched, or a suitable combination thereof.
The algorithm may perform an automatic thresholding operation which locates a reference surface in a reference box comprising image data from both the reference surface and the background.
The algorithm may perform an automatic thresholding operation which locates a reference surface in an image where the solder paste is deposited on the interconnect sites, and the resulting information may be stored.
The algorithm may perform xe2x80x9cblobxe2x80x9d or region analysis on the data resulting from the automatic thresholding operation, where the blob is formed by a segmentation operation to distinguish the background region from the reference data.
The blob analysis may include area, length, width, perimeter, aspect ratio or similar measures.
The blob may correspond to a portion of a conductor trace bounded by a reference box.
The blob may correspond to a region having substantially homogeneous grey-scale or height data, as often found for fiberglass and ground plane materials.
The image data may be supplemented with conductor coordinate information describing the location and/or connectivity of conductor runs separated from the interconnect. The algorithm automatically determines from image data, acquired prior to inspection, the position relative to the interconnect for placement of reference boxes from which the relative height of solder paste or components is to be determined. For instance, this may be provided as a xe2x80x9cGerber file, xe2x80x9d where the coordinate information is used to distinguish conductor image data from other circuit board materials (i.e. a xe2x80x9cknowledge basedxe2x80x9d segmentation algorithm).
At least one channel of height information is required in the inspection system but the image data for automatic learning and determination of box locations can be 3D, color or grey-scale information alone, or any combination thereof. The grey-scale information can be obtained from a standard video camera or from xe2x80x9cnon-conventionalxe2x80x9d imaging systems such as X-ray.
The preferred imaging system and method is disclosed in U.S. Pat. No. 5,024,529 where a laser scanner rapidly acquires both full field grey-scale and height (3D) information in perfect temporal and spatial registration. Also, the teachings of U.S. Pat. Nos. 5,546,189 and 5,617,209 can be used advantageously.
Prior art segmentation techniques as taught in U.S. Pat. No. 4,928,313 to Leonard et al. can be used for image processing.
The image data may be acquired by a laser system, but a video camera can be used for automatic determination of box locations from which height data is to be obtained during inspection.
Height data may be acquired as part of the learning process to improve reliability and robustness.
The video camera used for automatic determination of box (height reference) locations may also be used to obtain the height information.
The imaging system may comprise a plurality of sensors which are packaged as a single unit or separately, and which may acquire data either sequentially or in parallel.
The height data may be acquired using a triangulation sensor or alternatively, defocus or focus information, phase detection (Moire), or color encoding.
The 3D sensor may include a projector providing a point, line, grid, or other pattern onto the surface which is processed and decoded to provide height data.
The reliability of reference boxes may be qualified with the use of a consistency measure with other reference boxes in the proximity of an interconnect site. The reliability qualification may include analysis regarding the distribution of the reference boxes to verify that the reference surface computation at the position of the feature to be inspected is stable.
In determining the reliability of the reference data, a repeatability analysis may be performed wherein a best fit surface is calculated and the corresponding error distribution is analyzed.
During the learning process, and subsequent to the reliability analysis, instructions may be provided to the operator regarding preferred locations for the reference points so as to improve the stability and repeatability of the data
The results of the reliability qualification may be reported to the operator and interactively modified prior to inspection.
The learning algorithm may include steps where unsuitable reference locations are rejected, for instance by an xe2x80x9coutlierxe2x80x9d analysis which identifies locations which are not consistent with other data describing the reference surfaces.
The consistency measure may be deviations from a reference plane computed over at least three reference box locations containing height data.
Based upon consistency stability measurement, the system can automatically search in critical areas to identify alternate sites to automatically improve computational stability and/or consistency.